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ABSTRACT 

In an effort to improve the ways in which words are 
learned, the Johnson O'Connor Rt»search Foundation (JOCRF) is 
attempting to determine the difficulty level of all non-technical 
words in the English language. This item banking project entails: (1) 
identifying words t^at should be calibrated; (2) writing a test item 
for each word; ^3) testing the item in public schools and private 
schools; and (4; calculating a series of statistics to assess the 
relative difficulty of a word and pl«ce it on the JOCRF 's Vocabulary 
Scale. The vocabulary data base is composed of five data bases 
(ITEMS, USED, DISCUSS, STATS, and ALLSTATS) that are related to each 
other via various "key" fields. This report outlines the Rasch model 
statistical procedures used to determine the difficulty of a word. 
The rationale for using the Rasch model and a description of actual 
use of the statistical procedures are provided. The data base 
structure that the JOCRF uses to store the large quantities of 
statistical and verbal data generated by the project is also 
described. Eight appendices are provided containing numerous figures 
and tables that supplement the text. A 36-item list of references is 
ii -luded. (SLD) 
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Abstract 



The Johnson O'Connor Research Foundation is actively pursuing research to 
improve the ways in which one learns words. Li this regard the Foundation is 
attempting to determine the difficulty level of all nontechnical words in the English 
language. 

The item banking project entails several operations: the identification of 
words that should be calibrated, the writing of a test item for each word, the testing 
of that item in public and private schools, and the calculation of a series of statistics 
that assess the relative difficulty of a word and place that difficulty on the 
Foundation's Vocabulary Scale. 

This report outlines the Rasch-model statistical procedures that the Foundation 
uses to determine the difficulty of a word. The repc : gives both the rationale for 
using the Rasch model and a relatively nontechnical description of how the statistical 
procedures are actually used. The report also describes the database structure that the 
Foundation uses to store the large quantities of statistical and verbal data that are 
generated by the vocabulary project. 
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Introduction 



The Johnson O'Connor Research Foundation has had a commitment to the study 
of aptitudes and vocabulary acquisition since its founding in 1922. With regard to 
vocabulary, the Foundation maintains several ongoing programs: testing, education, 
and research. 

In terms of testing, examinees who take the Foundation's testing battery 
currently are administered Worksample 690, the latest in an extensive series of 
vocabulary tests, consisting of a total of 225 items, divided among three overlapping 
forms (easy, intermediate, difficult). A given examinee takes a short placement test 
(Worksample 695) and then the appropriate form of Worksample 690. The resulting 
raw score is converted to what is referred to as a Vocabulary Scale Score (VSS). The 
VSS value is the raw score that would be obtained on Worksample 690 if all 225 
items were administered (Statistical Bulletin 1980-33). The scale defined by VSS 
values is used by the Foundation as a common scale against which all vocabulary tests 
and vocabulary items can be referenced. The VSS scale also allows the vocabulary 
abilities of persons to be placed on a single continuum for comparison and norming 
purposes. Previously, examinees took all 225 items on Worksample 690. The use of 
the three overlapping forms eases the burden on low-vocabulary examinees, who took 
many items beyond their ability level on the longer test. The shorter forms also ease 
the burden previously placed on high-vocabulary examinees, who took many easy 
items that did not help to discriminate their vocabulary ability (Statistical Bulletin 
1980-33). 

In order to place future items on the VSS scale, Worksample 705-1 was 
designed to contain 75 of the Worksample 690 items, which were referred to as the 
"equating" items, and a group of new, easier items, which have come to be known as 
the Foundation's principal set of 60 "linking" items. Worksample 705-1 was 
administered to 212 junior and senior high school students in the spring of 1983. The 
difficulties of the 60 linking items were calculated and placed on the VSS scale. The 
60 linking items could then be used to "link" future experimental items to the VSS 
scale. These linking items were administered along with experimental items on 
subsequent high school testing series to link Ae new items to the Foundation's VSS 
scale (see Appendix D). 

In the field of education, the Foundation publishes a vocabulary building series 
known as Wordbooks (Bowker, 1979a, 1983). Each Wordbook contains teaching 
exercises for a group of 180 vocabulary words that fall within a nar/ow difficulty 
range, defined by the words' VSS values. One goal of the Foundation's vocabulary 
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research program is to determine the VSS values of all ilie nontechnical worus in the 
English language. These words will eventually be used in additional Wordbooks, By 
knowing the relative difficulty of all English words, the Foundation can suggest the 
words that are the most appropriate for teaching to a group of a given vocabulary 
abiUty. 



Equating Items to the Johnson O'Connor Vocabulary Scale 



In tenns of research, the Foundation now uses the Rasch measurement model to 
determine item difficulties and "equate" those difficulties with the VSS Scale (see 
Appendix B for a description of the Rasch model). There are numerous methods that 
can be used to equate tests and items within tests using the Ra.xh model. In this 
regard, Richard Gershon, Research Department Research Assistant, and David 
Sciiroeder, Research Manager, have conducted research that showed that for 
Foundation vocabulary items, there are no substantial differences between the 
commonly used equating strategies (see Appendix C; Gershon ife Schroeder, 1987). 
This research led us to the conclusion that "item anchoring" is the best n:tethod to use 
because it is the most iime-efficient and because it produces item files anil printouts 
with the equated item statistics. In brief, item anchoring is the process by which the 
values of the linking items are fixed at their VSS values in the analyses of the 
<!'^ficulties of the experimental items, so that no further equating is necessarj- (see 
Sch-u.:/, 1988, and Kelderman, 1986). 

The Foundation now uses the Rasch-model sortwaie for personal computers 
called BIGSCALE (Wright, Linacre, & Schultz, 1989) to calculate item statistics. 
Prior to the Worksample 741 test series, administered in 1989, a similar program 
called MSCALE (Wright, Congdon, & Rossner, 1987) was used. I will detail the 
procedure that we follow to analyze data with BIGSCALE: 

1. A raw data file that contains the answers chosen by examinees for a single 
vocabulary test form is constructed. Typically this form wili consist of 36 linking 
items (a reduced set of the original 60 items) and 74 experimental items. 

2. The linking items' difficulties are placed in a BIGSCALE-compatible "anchor 
file" with their predetermined values in "logit" units. Logits are the units that the 
Rasch model uses to express person abilities and item difficulties. Logit scales are 
desirable because they quantify the given variable on a linear, interval-level scale (see 
Appendix A). As noted earlier, the original anchor values for linking items were 
established when Worksample 690 was equated with the 7054 linking items, and the 
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use of these anchor values ensures that all new items tested by the Foundation are 
placed on a common scale (see Appendix D for a complete overview of the linking 
structure). Beginning with Worksample 738, the set of linking items was reduced 
from 60 items to 36. Two additional sets of linking items were constructed for 
Worksample 741, one set for use in early primary grades an J the other for use with 
college students. The list of the three linking, item sets and their anchor values can be 
found in Appendix D. 

3. The data set is analyzed with BIGSCALE (Wright, Linacre, & Schultz, 1989). 
The printouts and item statistic files are geneiated so that the item measure is 

ai tomatically placed on the Foundation scale without further 'xjuating.* A sample 
program is shown in Appendix E. A saiTiple printout of the results is in Appendix F. 

4. The difficulty value, or "logit measure," for each item (and person) 
correspDnds to the vocabulary ability level at which an examinee has a 50% 
probability of getting the item correct. Logit measures can be converted to VSS units 
by the following linear transformation: 

(MEASURE X 26.78) + 128.40 

Since the Wordbook program uses items at the 80% level, however, the formula 
appropriate for conversion to 80% is as follows: 

(MEASURE X 26.78) + 165.40 

The first formula was derived by regressing the VSS values of a group of 
Worhsample 690 items on the logit measures obtained for those same items. This 
method determined that each logit represents 26.78 VSS units. In the Rasch model 
the point representing the 80% chance of getting an item correct is always 1.38 logits 
from the 50% point. Multiplying this by the logit size given in the first equation 
results in the second equation. 

Although we have used these linear formulas, it may be the case that a single 
linear transformation is not accurate across the entire VSS range. Further research 
should be conducted to determine whether this is the case and whether additional 
formulas need to be derived. 



'EIGSCALE also allows for the one-step computation of person ability estimates 
on the Foundation VSS scale expressed in Rasch logits. These estimates are 
computed at the same time that the items are calibrated and are provided along with 
person fit statistics. 
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5. After VSS values are obtained, it is necessary to determine the quality 
(validity) of each item. A good-quality item is one for which people below a given 
levd consistently get the item wrong, while people above the level consistently get it 
right. It is possible, for example, that a poor-quality item was guessed correctly by 
an oitremely large number of low- vocabulary people, or that one of the msleads was 
so attractive that people who would ordinarily have blown the word answered that 
item incorrecdy. Fortunately, JilGSCALE provides several statistics that make it 
fairly easy to determine the qu ality of the item. The most important of these statistics 
are called INFIX and OUmT. Fit statistics serve much the same function in item 
response theory as the item-fx)tal coiielation in classical test theory.^ They provide a 
measure of how well an item agrees with the total test score. 

In general, a fit value near zero mdicates an average degree of agreement between 
the iton and the total test score. A negative value indic-ites a bemr than average 
degree of agreement bettveen the item and the total test score. The lower the negative 
value, the greater the level of agreement. A positive value indicates a poorer than 
average degree of agreement between the item and the total test score. The higher the 
value, the lower the level of agreement. 

In other words, im item has a negative fit if persons with word knowledge better 
than the item difficulty almost always get the item correct, and persons with word 
knowledge below that of the item difficulty almost always get the item wrong. For 
the majority of items the fit value is near zero (plus or minus two), indicating that the 
more-competent persons usually answered the item correctly, and the less-competeni 
persons usually answered the item incorrectly. Positive fit statistics indicate that at 
least one of the two conditions was not met-that is, a relatively large number of 
higher-vocabulary persons answered the item incorrectly and/or a relatively large 
number of lower-vocabulary persons answered the item correctly. 

5a. Item INFIT is roughly equivalent to the ability of an item to accurately 
discriminate in the vicinity of its difficulty level. In VSS terms, it means that for an 



^Classical test theory was firit presented by Charles Spearman. He posited that 
test scores were actually the sum of two components: the person's "true" score plus 
an error component. Using cla'isical methods, a person's measure on a given variable 
is solely determined by his total test score. The problem with this approach was that 
two good-quality tests could be constructed, but if one consisted of easier items than 
the other, the results of the tests would differ and would not be directly comparable. 
In addition, the estimated difficulty of the test was directly related to the ability of the 
sample taking the test, with no direct method of relating the same test to a more- or 
less-able sample (Mislevy, 1990). 
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item of VSS 100, only people with a VSS score of 100 or more should answer the 
item correctly. In some situations, such as in the use of linking items used to equate 
populations, highly negative INFTT values would be undesirable (personal 
communication, Benj.aroin Wright, April 26, 1990). However, for the Wordbook 
testing program, negative INFTT values are probably just fine.^ (As noted, items that 
are used, for test linking should probably not have INFTT values greater than + 2.0.) 
Many people + 2.0 as their cutoff for item INFTT, but since our samples are 
usually relatively lar^e (i.e., 400-500 students), we reject only items with INFTT 
values greater than +4.0, Items with INFTT values of this magnitude should be 
rewritten and retested. 

5b. Item OUTFTT is similar to INHT, but it is more sensitive to unexpected 
correct responses by low-vocabulary examinees and to incorrect responses by high- 
vocabulary examinees who are far above the VSS level of the item. Items with 
OUTFTT values greater than +4.0 should be rewritten and retested. Oftentimes a 
large OUTFTT value is obtained when a large number cf low-ability people guess an 
item correctly. This may be due to such things as the item being too difficult for the 
sample population (see also Point 6), or to the misleads being so unattractive that they 
were never chosen. (Items that are to be used for linking also should not have an 
OUTFTT value more than 2.0 units away from the INFTT value; this may indicate 
guessing or order effects* for the item.) 

5c. The Mean Square statistics provide additionrl measures of the quality of an 
item. They refer to the ratio of the item variance that actually occurred to that which 
was expected, given the item measure that was obtained and the ability levels of the 
people in the sample. The maximum value of Mean Square that should be acceptable 
for INFTT is 1.2, meaning that a maximum of 20% of the item variance is 
unexplained by the model. BIGSCALE also produces a value for Mean Square 
OUTFTT. Based on the results of the Worksample 741 data analysis, it would appear 
to be reasonable to select a maximum acceptable Mean Square OUTFTT value of 1.4. 

6. The final issue that must be addressed is whether the item was given to the 
correct population. If the Uem was much too easy or much too difficult for the 



^Negative fit values are satisfictory except in the case where the ability of the 
sample differs substantially from the difficulty of the iten: (personal communication, 
Benjamin Wright, May 1989). Because of this, the bad sample criterion should 
probably be made more sirict for items with high negative INFTT. 

*An order effic! occurs when an item's difficulty changes depending on it; 
position in a test. 
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persons who took St, the difficulty estimate for that word will not be accurate. Items 
that are eliminated for either of these reasons are not necessarily "bad" items, but 
they must be retested with an appropriate population. The use of estimates of word 
difficulty, such as those m The Living Word Vocabulary (a word list th£», includes 
over 40,000 words and gives a percentage score for knowledge by persons of varying 
grade levels; Dale & O'Rourke, 19S1), limits the number of items that must be 
retested. 

The acceptable difficulty range for items dqpends on the sample population. Any 
item that is more than plus or minus 2.5 logits from the mean person measure for that 
form should be retested. The mean person measure for a form can be found in Tablt 
20 of the BIGSCALE printout for that form (see Appendix F). 

I have described se, „al special considerations to be used in selecting linking 
items (see Points 5a and 5b). I would further suggest that no short explanation would 
sufficiently cover all the contingencies and issues that may arise in selecting linking 
items. For example, linking items should adhere to stricter selection criteria 
regarding fit statistics than should other vocabulary items. Otherwise, an item that 
appears to bo of good quality when administered to low-ability examinees may end up 
being of poor quality when administered to high-ability examinees. When this occurs, 
the quality of the linking deteriorates, and new experimental items that are presumed 
to be )n-ecdy linked to the Vocabulary Scale will have inaccurate difficulty values. 
Tlierefore, linking items should probably not be selected without the aid of someone 
well-versed in the Rasch model. Since 1983, the Research Depailnent has received 
consultation from Dr. Benjamin Wright regarding our choices of linking items, 
equating procedures, and Rasch-model software. 



The Vocabulary Datzbase 



The vocabulary database is composed of five databases that are related to one 
another by means of various "key" fields. (A relation in computer software refers to 
the capacity to look something up in one file and automatically be able to find related 
information in other files.) The structures of tlic five databases can be found in 
Appendix G. The test series covered by the databases can be found in Appendix H 
The following is a description of the databases and how they can be used: 

ITEMS This database is a collection of all the items that have been 

calibrated, whose test words are available for use in future 
Wordbooks. 
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USED 



This database contains all the vocabulary items ever tested by the 
Foundation. 



DISCUSS This database contains Wordbook discussions, pretest items, 

exercises, and review test items for future use in Wordbooks. Many 
of the Worksample 705 and 722 words have completed records in 
DISCUSS. (Technical note: the memo fields from DISCUSS.DBF 
are actually maintained in DISCUSS.DBT. This file should not be 
erased.) 

STATS This database lists all the item statistics that have been computed for 

Foundation items. This database should be ruaintained by the 
Research Department as a statistical archive. Although item 
difficulty values are contained in this file, the VSS value for each 
item can also be found in ITEMS. STATS contains items ftom test 
series where the only statistic listed may be the VSS value. 

ALLSTATS A second databi'^e similar to STATS is ALLSTATS, which contains 
items for which we have Rasch-model measurement statistics. A 
recent list of items with accq)table statistics v/as published by the 
Foundation in 1988 (Technical Report 1988-3). 
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Appendix A 



The Rasch Model and the Item Characteristic Curve 



The Teacher's Manual of the Wordbook series outlines the method originally used 
by Richard Bowker of the Foundation for determining item difficulty (Bowlar, 1979b, 
pp. 4-6). In brief, a subset of the Worksample 690 items was administered along 
with the experimental items. Bowker then graphed the proportion of persons ivho 
answered the item correctly for various score ranges on the Worksample 690 sub^t. 
The difficulty of the item was defined as the Vocabulary Scale Score where 80% of 
the examinees answered the item correctly. "When Wordbooks 7 and 5 (Bowlffir, 
1983) were added to the series, Bowker began to us2 the Rasch measurr-'^ent program 
BICAL (Bowker, 1982; Wright, Mead, & Bell, 1980). His use of BICAL was stiH 
graphical in nature, however, as he relied on examination of the BICAL charts to 
determine the item's difficulty. 

Beginning with the 705 series, 'dia Foundation began to use the Rasch model 
statistics themselves (see Appendix B). The Rasc*^ model employs a theoretical curve 
for each item that relates the percentage of persons who answer an item correctly to 
the log-linear difference between their ability and the difficulty of the item. The 
"Theoretical" curve presented in Figure 1 shows the likelihood of a person answering 
an item correctly given the distance of that person's vocabulary knowledge from the 
item difficulty (in logits). You will note that when the distance is zero (the person's 
ability is the same as the item's difficulty), the person has a 50% likelihood of 
answering that item correctly. As described in the equating section of this report, 
however, the Foundation defines the difficulty of an item as being the point where 
80% of the persons answer the item correctly. In logit terms this is equivalent to 
adding 1.3863 logits to the difficulty of the item. 

Figure 1 also shows the actual percentages obtained using Worksample 741 
compared to the theoretical values predicted by the Rasch model. The "Actual Wks. 
741" line indicates the average results obtained for Worksample 741 across 22,000 
students taking 110 items each. The "Theoretical" line indicates the percentages 
suggested by the Rasch model. As you can see, there appears to be satisfactory 
agreement between the two methods at all levels except for extremely low-ability 
persons, for whom chance becomes an issue (theoretical values below 20%). 

Given the close agreement between ihe Rasch model and the observed likelihood 
function for vocabulary items, a clear case can be made for using tlie Rasch model 
(also see Appendix B). The Rasch model allows a single pass to be made of the data, 
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Figure 1 

Comparison of Actual versus Theoretical 
Item Characteristic Curve of Wks. 741 



Likelihood of Correct Answer 




Distance of Person from Item Difficulty 



Actual Wks. 741 Theoretical 

9 



using a computer program such as BIGSCALE (Wright, Linacre, & Schultz, 1989) to 
simultaneous^ compute the i>erson abilities and the item difficulties. It should be noted that 
this approach uses all the data to estimate item difficulty, whereas the graphical approach 
outlined by Bowker makes no use of data away from the determined difficulty level. The 
Rasch approach liso uses a smooth curve thai corresponds closely to the actual probabilities, 
using a linear (interval-level) scale (Wright, 1977a). 
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Appendix B 



Constructing a Rasch Item Bank 
for the Johnson O'Connor Vocabulary Tests 

Rich&rd Smith 



Recently, uiere has been a gi-^at deal of interest in applying latent trait theory to test 
development research. Latent trait models are useful because they provide a way of 
analyzing and interpreting responses to items independently of the ability of tlte sample used, 
llasch recognized that objective measurement requires i^rson measures that do not reflect the 
particulars o^ 'he items used. Furthermore, the ordering of the items that define a variable 
should h's u.J'?pendent of the persons measured (Rasch, 1960, 1961, 1966a, 1966b, 1977; 
Wright, 1968, 1977b). 

The primary task of psychological measurement is to ascribe meaning to scores in 
such a way as to establish a joint order of persons and items along a single common linear 
scale. To measure and understand individuals, we must construct person-item interactions 
that provide insight into the degree to which a person possesses the aptitude. 

Tentatively we may consider what happens to the probability of a person succeeding 
on a test item. The Rasch model has only one ability parameter B for each person and onl;. 
one difficulty parameter D foir each item i. The probability of a right answer is determined 
by the difference between ability and difficulty (B-D) expressed as a ratio of natural logs. 
Persons with more ability should always have a greater probability of answering any item 
correctly than persons with less ability. Easy items should always be answered correctly 
more often by everybody than hard items. If the response person n gives to item i is 
expressed as Xai=l for a correct response, and X^=0 for an incorrect response, then the 
Rasch model for measuring persons and calibrating items becomes: 

P[Y =1]=-^ 



The Rasch model is the only latent trait model where the unweighted sum of right answers 
given by a person is a sufficient statistic for the person's ability. This means that the 
conditional probability of the item responses of an examinee, given the person's raw score, is 
independent of the examinee's ability. Similarly, the unweighted sum of right answers given 
to an item will contain all the information necessary to calibrate that item along the variable. 
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The uniqueness of the Rasch model focuses on the coi^cept of specific objectivity. 
This is formalized by Wright as test-free person measurement and sample-free item 
calibration. Objectivity involves logical order, parameter separation, and estimation 
efficiency. The property of logical ordering implies that for any person, the probability of 
success is greater for an easy item than for a hard one; for any item, an able person has a 
greater probability of success than an unable one. The ordering of every person and every 
item along a single common variable allows objective comparisons among persons and items. 

A basic requirement for Rasch measurement is that the variable L3ing measured is 
unidimensional, so that a single score is meaningful and useful. This has many practical 
implications. It means that a person's ability is all that is needed to predict his performance 
on a set of test items. It is not nj^essary to know anything about what group the person 
belongs to or what year the person took the test. It also means that all persons moving in the 
same direction along the line of the variable must pass through the same points in the same 
order. We can make probability statements about any person encountering any item, based 
on an estimated ability and difficulty. The items that are ordered along the line provide the 
operational definition of the variable. The relative positions of items on the line are 
determined by the performances of persons on those items. 

Although the p-.w score is a sufficient statistic, it must je transformed into a linear 
and objective measure of the person's position on the variable. The logistic transformation 
stretches the score at the extremes so that the resulting logits are linear in the ability implied 
by tfie score. Linearity means that an increase of one unit represents the same increment in 
ability at any point along the scale. Although the raw score is specific to the test, the logit 
measure is general on the variable. 

In summary, the structure of the Rasch model in which parameters enter linearly 
without interactions makes the complete separation of, the model's parameters possible. As a 
result, the likelihood equations can be written so that it is possible to derive conditional 
estimation equations for person abilities and item difficulties that are completely independent 
of each other. 

Separable person and i^en^ parameters permit the calculation of sufficient statistics that 
are simple counts. These are the n- nber of right answers for each person and the number of 
successful persons for each item. S.nce all information about the abilities of the persons is 
contained in their raw scores, the estimation equations for the item difficulties can be 
expressed in terms of the unknown difficulties and the observable person scores. These 
sufficient statistics correspond to the greatest data reduction that can be achieved while still 
defining the likelihood. 
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The essential aspects of specific objectivity cannot be separated from each other. A 
unique ordering of persons and items to be inferred f/om the data is crucial. This inference 
requires a probabilistic Rasch measurement process which has separable parameters and 
hence sufficient statistics. These properties and their psychometric implications are described 
in Rasch (1960, 1968), Andersen (1970, 1973, 1977), and Wright (1968, 1977a, 1977b, 
1985). 

Item Bank Building 

An item bank is a collection of carefully calibrated test itemo that define a variable. 
Ic is a continually evolving measurement system in which the systematic assessment of 
educational achievement or acquired knowledge is a permanent activity. This section 
describes several features of item banks, including the motivation for banking techniques. 
The primary incentives that justify the effort required to establish and maintain a Rasch-based 
item bank are meaning and convenience. Meaning comes from the careful delineation, over 
a broad range of application, of the variable that the items in the bank are designed to 
measure. Convergence comes because a calibrated bank iTiakes it easy to construct and 
equate new forms for a variety of purposes. 

A clear, unequivocal, and objective definition of the variable to be measured is 
fundamental to the success of any measurement task. For the Foundation, it must be 
possible to imagine that the particular knowledge or aptitude of each examinee can be 
described quantitatively on a scale. Apart from the quantitative attributes of such a scale, 
any meaning that is to be attached to it must come from tiie items that are used to observe it. 
It is only tiirough the placement of items along the continuum according to their relative 
difficulty that we can understand what it means for an examinee to be at a particular location 
along the continuum. Once the items are located, tiie bank is, in principle, built. 

An essential psychometric quality of Rasch item banking is tiiat when items are 
calibrated onto a common variable, each item represents a position on the variable that is 
also represented by other items of comparable difficulty. This makes it possible to infer an 
examinee's mastery witii respect to tiie basic variable tiiat the items share, regardless of 
which items are administered or whom else has been tested (Wright & Bell, 1984). Each 
person's position on tiie variable places tiiat persor. among whomever else has ever taken any 
set of items from the bank. For example, examinees will receive scores tiiat are 
commensurate with tiieir current knowledge of English vocabulary, irrespective of which 
items from the bank are used to assess their knowledge. 

The most fundamental part of developing an item bank is to objectively define tiie 
variable and to locate ite.ns along a line according to their relative difficulty. When a 
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variable is mapped in terms of its items, then standards can be established and meaning 
attached to being at a particular point along the variable. 

After all items are calibrated onto a common linear scale, any subset drawn from the 
bank will be automatically equated to the bank and to any other possible set of bank items. 
This is achieved without any further testing. As a result, it is simple to equate tests from 
year to year or to equate multiple forms given on the same occasion. Scores a person makes 
from time to time are directly comparable and the rate of progress apparent, Choppin (1978) 
provides a comprehensive examination of the conceptual issues and psychometric implications 
of item banking and item calibration. 

Since many persons do not follow our expectations of which items are easy and which 
are hard, we can apply Rasch's probabilistic model to impose an orderly response process on 
the data (Wright & Bell, 1984), In order to have a common basis for describing progress, 
there should be agreement among researchers and examinees as to which items are hard and 
which are easy. 

Several steps are necessary to build and maintain an item bank: 

1) Designing test forms. 

2) Calibrating test forms, 

3) Analyzing fit, 

4) Linking pairs of forms, 

5) Calibrating forms on the bank, 

6) Analyzing link fit, 

7) Controlling item quality, 

8) Monitoring and updating the bank. 

First, items are written and distributed among test forms so that there is a network of 
common items that is practical to the testing situation. Forms are designed and administered. 
The process of calibrating sample-free item difficulties is performed under the expectation 
that these data can be used to approximate additive conjoint measurement (Brogden, 1977), 
The calibration of items that is sample-free and the measurement of persons that is test-free 
are the precious ingredients that supply the natural fuel needed for the development of a 
successfiil item bank measuring system. 

The computer program BICAL (Wright, Mead, & Bell, 1980) [Note: more recenUy, 
MSCALE (Wright, Congdon, & Rossner, 1987) and BIGSCALE (Wright, Linacre, & 
Schultz, 1989)] was used to derive estimates of person abilities and item difficulties and to 
test the fit of items within each of the vocabulary tests. These item difficulties are invariant 
with respect to the ability of the calibrating sample; however, they are defined by the center 
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of the items in the specific examination. Aji item will appear to have a different difficulty 
for each test in which it appears, so we must adjust all difficulties on all exams so that tliey 
are portioned relative to one common origin. This requires linking together all relevant teii* 
by calculating translation constants that shift these items to a common bank reference scale. 
The technique implies that if test X and test Y share a common set of K items, called the 
link items, the difficulty scale of test Y is adjusted to the scale of test X. Therefore, the link 
between two tests is estimated by the difference between the difficultie" of any item 
calibrated in both exams. Common items between any pair of forms provide a. duect 
estimate of the relation of the two forms. If the common it?ms and the other items in both 
tests fit the Rasch model and Lre calibrated on the same latent variable, this method yields a 
pool of calibrated items whose estimated difficulties are on a common scale with a common 
linear metric. 

After the bank has been constructed, it will need constant monitoring to verify that no 
item has lost its effectiveness. If some items are becoming too familiar or have been used 
too much, these items can be simply removed from the bank without disturbing the other 
items. An item difficulty is estimated every time an item is administered. When this 
estimate ir : tistically different from the item's bank difficulty, then thought must be given 
to what may have caused this change and how to resolve it. Whenever a new exam is given 
that uses items from the bank, it will be necessary to calibrate the new form and determine 
the appropriate translation constant to link the new exam to the existing bank through the 
reused items. New items must be introduced into the bank in the same way the original 
items were established when the bank was created. 

Wright & Stone (1979, Chapters 5 and 6) describe procedures for calibrating tests and 
constructing item banks using the Rasch model. Further issu&s concerning the curricular 
implications of item banking and the psychometric basis of banking, along with computer 
programs and equations for accomplishing banking are presented in Wright & Bell (1984). 
Millman & Arter (1984) discuss the vast array of item bank features thai allow them to 
operate effectively within diverse instructional and assessment environments. 



15 



ERIC 



Appendix C 



Unking Constants Obtained Using Various Equating Strategies 



Numerous equating strategies are suggested in the literature of item response theory. 
To select an equating strategy for the Foundation's vocabulary item banking projeC five 
commonly used methods were carried out for three test forms: Worksample 705-2, 
Worksample 722-5, and Worksample 722-9. 

Method 1 is the simplest. One simply averages the difficulties of the 60 linking items 
and computes the difference between that average and the average for the same items when 
adniirJstered on Worksample 705-1. 

Method 2 employs a complex set of spreadsheet calculations to limit the set of linking 
items to those with standardized residuals below a paracular level (see Wright & Stone, 
1979, for a complete description). Method 2a gives the linking constant obtained when the 
residuals of the linldng items were limited to a maximum value of 3. In other words, after 
the calculations are carried out, some of the linking items are discarded for the given form 
because their standardized residual values are greater than 3. The difficulties -obtained on the 
remaining items are averaged, and the linking consta-^t is computed as the difference bet-veen 
this average and the average of the same limited set of linking items administered on 
Worksample 705-1. Method 2b is similar to 2a except that the residual requirement is 
stricter and the retained Jip'dng items must obtain values less than 2. This results in an even 
smaller set of linking items being used. 

Method 3 appears to be similar to Method 2 in that first the 46 best linking items 
were selected from the Worksample 705 test series using a standardized residuals analysis 
that included all the Worksample 705 test series forms. Although all 60 linking items were 
left on the test, the difficulty values of the 46 items were anchored in the MSCALE analysis 
of each form. While the previous two methods required MSCALE to be run, and then a 
linking constant to be added to the item difficulties obtained, item anchoring within 
MSCALE allows the printouts to include item difficulties already linked to the Foundation's 
Vocabulary Scale. It should be noted that none of the anchored items were deleted from the 
analyses. 

Method 4 is the same as Method 3 except that the anchor items were selected from a 
spreadsheet residuals analysis of all the Worksample 722 forms. 
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^ Table 1 shows the results of the above linking and anchoring strategies when applied 
to each of three test forms. The values within the table are the effective linking constants for 
the various metf ods. 



Table 1 



Anchoring Stratppv 


JSJj-i 


TOO C 

722-5 


'TOO A 

722-9 


1) All 60 705-1 linking items 


-3.336 


-2.658 


-2.250 


2) Custom spreadsheet selection 
a) Standardized residuals < 3 


-3.295 


-2.69 


-2.263 


b) Standardized residuals < 2 


-3.288 


-2.67 


-2.276 


3) 46 preselected 705-1 links 


-3.319 


-2.668 


-2.264 


4) Best 32 links chosen from 
across all 722 forms 


-3.346 


-2.673 


-2.294 



Simple observation leads one to conclude that the differences between the above 
methods are negligible. This led us to conclude that item anchoring was the superior linking 
strategy because it is the most time-efficient and results in MSCALE outputs that already 
place all the items on the Johnson O'Connor vocab' 'ary scale. 
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Appe&dbcD 
Linking Structure, Items, and Anchor Values 



Worksample 705-1 



^ . "... 



60 



EquatK*3 Items 



Linking Items 



Worksample 705 Forms 2-10 
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Worksample 722 Forms A-J 



722D 722C 



722E 



722B 



722F 



722G 




722A 




New Items 



705-1 Linking Items 



722H 7221 



Worksample 722A 



Worksample 738 Forms A-M 

(Only Forms A-J shown) 



7360 738C 



738E 



738B 



738F 



738G 




738A 




N«w Items 



700-1 Linking Itomt 



738H 7381 



Worksample 738A 
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Origins of Wks. 741 Linking Items 



Medium (rorn« 99-48) - 



Hard (Form* 80-81) - 



Eauy from Bank IWorktampI* 706*1 Workeampio eoo 



Worksample 741 Forms 1-29 

(Only Forms 1-10 shown) 



741-4 741-3 



741-6 




741-1 




New Ite^ns 



Eaoy Linking Items 



741-8 741-D 



Worksample 7h-1-1 
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Worksample 741 Forms 30-49 

(Only Forms 30-39 shown) 



74V32 



74V35 



741-36 



741-37 



741-31 




741-30 



741-39 




New ittrr.e 



Mtdlum Linking Items 



741-38 



Worksample 741-30 



Worksample 741 Forms 50-61 




741-50 




N«w Items 



Haro Linking items 



Worksample 741-50 
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For the following lists, all anchor values are expressed in logits on the Foundation's 
Vocabulary Scale (at the 50% level). 



Current Easy Linking Items and Anchor Values 



TOPIC 


-5 60 


SHALLOW 




NURSED 




GAP 


•4 56 


COZY 




CABLE 


-4 11 


GUIDE 


-5 22 


NATURAL 


-5 65 


GNAWED 


-4 97 


JOURNEY 


-5 ^^5 


COMPRESS 


-4 45 


GRAVEL 


-5 09 


SILv vEL 


-4 33 


GRAiTTED 


-4 30 


FLCCK 


-4 56 


CRAM 


-4 56 


COMMOTION 


-4 16 


DECLARE 


-4 23 

^^•X»»/ 


WITHDRAW 


-4 02 


VALUE 


-4.15 


PRECISE 


-3.75 


APPROPRIATE 


-3 77 


GRIEF 


-3.61 


GLOBAL 


-4.72 


SLAY 


-3.52 


FRACTION 


-3 59 


BUREAU 


-4.10 


INFURIATE 


-2.94 


POURED 


-3.86 


BRISK 


-2.85 


UNIFORM 


-3.68 


EMPHASIZE 


-2.83 


UNSAVORY 


-2.71 


MUDDLE 


-3.33 


CONCEITED 


-2.58 


CURVATURE 


-4.01 
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Current Intermediate Linking I^ms and Anchor Values 



VANloH 


-5.80 


CTTDTTTTCT 


-4.33 


Al>QlTOT\ 

AooUKD 


AHA 

-4.14 




-4.78 


ArrKUrKiA 1 ii 


-3.77 


rr A XTV 


-4.59 


Q1TDTCV 


-4.D3 


CUMMU 1 iUvi 


-4.16 




-4.33 


A CC AT7T T* 


-4.22 


TMTFCDXT A T 


-4.06 




-3.75 


WJbAKI 


O OA 

-3.89 




-3.61 


rUoonoo 


-3.94 


CT A V 


-3.52 


ivlulD 


-3.54 


TVTCK if A T 

DloMAL 


-2.90 


l^IvIrJtlAolZj^ 


-2. S3 


"D A 'D'DTr' A T\TJ 
i^AKKiv^AliC 


-3.34 




-2.85 








O C\A 

-2.94 


TJYTTOPP AlSJT 


-2.03 




-2.2o 


UiNoAVUJvI 


-2.71 




O CO 

-2.58 


"DT T3X>fTCU 


O AO 

-2.48 


liNv^UloillVJD 


-2.16 


PUTRID 


-2.08 


SERENE 


-2.03 


CLAMOR 


-2.26 


MEAGER 


-2.34 


ABHOR 


-1.83 


MONUMENTAL 


-1.92 


ACKNOWLEDGE 


-2.03 
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Current Difficult Linking Items and Anchor Values 



RIGID 


-3.54 


BARRICADE 


-3.34 


EXUBER/JxTT 


-2.83 


CONCEITED 


-3.56 


BLEMISH 


-2.48 


MEAGER 


-2.34 


LLAMUR 


-2.26 


ACKNOWLEDGE 


-2.03 


MONUMENTAL 


1 AA 

-1.92 


ABHOR 


-1.83 


SOUVENIR 


-3.94 


AGHAST 


-2.56 


REPLICA 


-3.94 


AGITATED 


-4.63 


RESPONSIVE 


-3.26 


DETESTED 


-3.31 


TERMINATION 


-5.52 


INCISION 


-4.35 


PROLONCj 


-4.79 




-2.92 


DELUMONo 


-3.56 


ollMULATED 


-3.94 




-2.20 


RbrULolVJb 


-3.20 


DBTERIORATED 


-1.16 


BESEECHES 


A Af 

-0.25 


TTT TTX/rA'mxjr 


O AA 

-2.09 


T ntCTTDCT V 

LbloUXELY 


O A^ 


RIGOR 


-1.84 


SANCriTY 


-1.32 


SCRUPULOUS 


-0.06 


SUBORDINATE 


-:i.27 


CAPRICE 


-0.41 


ASSUAGING 


0.61 


EFFRONTERY 


0.49 


ERUDITE 


1.78 
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Previously Used Linking Items and Their Anchor \'alues 
(used with Worksamples 705 & 722) 



ABSURD 


-4.14 


SACRED 


-3.14 


ABUNDANT 


-3.61 


SLAY 


-3.52 


APPROPRIATE 


-3.77 


TRIBUTE 


-3.21 


ASSAULT 


-4.22 


ZANY 


-4.59 


COMMOTION 


-4.16 


EMPHASIZE 


-2.83 


DISPUTE 


-3.61 


BADGER 


-3.05 


VANISH 


-5.79 


FEEBLE 


-3.56 


GRIEF 


-3.61 


NONCHALANT 


-2.19 


TASK 


-4.78 


SERENE 


-2.03 


EXTERNAL 


-4.06 


BLEMISH 


-2.48 


PRECISE 


-3.75 


COLOSSAL 


-3.49 


SHRIEK 


-4.53 


FOE 


-3.36 


SHRIVEL 


-4.33 


RESIDE 


-2.21 


BAPJRICADE 


-3.34 


UNSAVORY 


-2.71 




-2.58 


ACKNOWLEDGE 


-2.03 


CONCEITED 


-2.58 


COMBUSTION 


-2.76 


WEARY 


-3.89 


INQUISITIVE 


-2.16 


BARTER 


-3.01 


PERPETUAL 


-2.87 


BRISK 


-2.85 


PETTY 


-1.77 


COMMEND 


-2.36 


RIGID 


-3.54 


CONSUME 


-3.16 


PUTRID 


-2.08 


CONTEOT 


-3.65 


OGRE 


-3.18 


DEVASTATE 


-2.26 


PERILOUS 


-2.83 


EXUBERANT 


-2.83 


MEAGER 


-2.34 


DISMAL 


-2.90 


MONUMENTAL 


-1.92 


HEX 


-4.33 


ABHOR 


-1.83 


INFURIATE 


-2.94 


DISCLOSE 


-2.00 


INVINCIBLE 


-3.84 


VALOR 


-2.29 


OBSTRUCT 


-2.55 


AMIABLE 


-1.92 


POSSESS 


-3.94 


CLAMOR 


-2.26 
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Appendix E 

Sample BIGSCALE Command File 
(used for Worksample 741, Form 1) 



&INST 
Ni\MEl=l 

n:-:=iio 

nEMl=6 

TtTLE='M741-r 

MSCDAT='F:M741-1.DAT' 

TABLES ='00001 101 lOOlOOOOOOOr 

IFILE='M741-1.ITM' 

PFILE='M741-1.PER' 

AFILE='EASY741.ANC' 

XWIDE=1 

CATEGS=5 

CODES ='12345* 

KEY1='2131322512452455354341124552354413132124151534451551233323 

3414434115355533543411245555313114241135213311325155' 

ENDIT=20 

&END 
TOPIC 
SHALLOW 
NURSED 
GAP 
COZY 
CABLE 
GUIDE 
NATURAL 
GNAWED 
JOURNEY 
COMPRESS 
GRAVEL 
SHRIVEL 
GRANTED 
FLOCK 
CRAM 

COMMOTION 
DECLARE 
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WITHDRAW 

VALUE 

PRECISE 

APPROPRIATE 

GRIEF 

GLOBAL 

SLAY 

FRACTION 

BUREAU 

INFURIATE 

POURED 

BRISK 

UNIFORM 

EMPHASIZE 

UNSAVORY 

MUDDLE 

CONCEITED 

CURVATURE 

SECURE 

DWARF 

BLOW 

DRAW 

POOR 

JUMP 

LITTLE 

FIRE 

AIR 

YANK 

LOVE 

CLASS 

CHURCH 

MARKET 

GATE 

ABOUT 

GROUND 

LADY 

PLUS 

SUM 

FINISH 

CAPTAIN 

DIVIDIS 



PIPE 
SAVE 
NAP 
MIDDLE 

p/'m 

FREEZE 

APARTMENT 

RUN 

MUD 

DANCE 

MAIL 

ABOVE 

EXPLAIN 

EXPLORE 

SLIDE 

BANK 

JOKE 

FOLLOW 

HANDSOME 

CHASE 

EARTH 

CALL 

FEAST 

DROP 

DARK 

PAY 

ACROSS 

KITCHEN 

PLATCr 

SHOOT 

GARDEN 

GLOW 

ACT 

FACE 

AGE 

WK6CK 

SADDLE 

TURN 

JOB 

STORY 

FIELD 



ALARM 

BIG 

STONE 

JOIN 

HALL 

SLEBtY 

BRCX)M 

DRY 

BEHIND 

END NAMES 



Appendix F 

Sample BIGSCALE Output 
(Worksample 741, Form 1) 



BIGSCALH 
A mCH PROGRAM FOR <!ATING SCI' AMALYSIS 



PERSON KEASUREMENT, ITEH AMD STEP u IBRATION 
WITH PERSOM AKO ITEM FIT ANALU'IS 



DIRECT ENQUIRIES TO: 

BEMJAHIH D. URIGHT 
KHSA PRESS 
583S S KIHBARIC AVE 
CHICAGO ILLINOIS 60637 

(312) 702-1596 
(312) 288-1762 

COPYRIGHT (C) BENJAMIN D. WRIGHT, 1989 
VRITTEM BY BENJAMIN D. WRIGHT, JOHN M. LINACRE, AKO MATTHEW SCKULTZ 

JANUARY 1990 VERSION 1.53 



CONTRCM. VARIABLES 



ESSENTIAL TABLES 



ADDITIONAL TABLES AVAIUBLE 



3. CATEGORY PROBABILITY CURVES 

4. HOST PRC^ABLE RESPONSES 



t. DIAGNOSIS OF MISFITTING PERSONS 



12. ITEM CALIBRATIONS IN ENTRY ORDER 
17. PERSON MEASURES IN ABILITY ORtER 



20. PERSON, ITEM AND STEP Sl^RY 
TITLES M741-1 



5. PERSON AND ITEM DISTRIBUTION MAP 

6. ITEM MAP BY NAME 

7. PERSON MAP BY NAME 

2. DIAGNOSIS OF MISFITTING ITEMS 

8. ITEM PLOT OF INFIT VS. DIFFICULTY 

9. ITEM PLOT OF OUTFIT VS. OIFrlCULTY 

10. PERSON PLOT OF INFIT VS. ABILITY 
•1. PERtOH PIOT OF OUTFIT VS. ABILITY 

13. ITEM CALIBRATIONS IN DIFFICULTY ORDER 

14. ITEH CALIBRATIONS IN INFIT ORDER 

15. ITEM CALIBRATIONS IN ALPHA ORDER 

16. PERSON MEASURES IN ENTRY ORDER 

18. PERSON MEASURES IN INFIT OPJiER 

19. PERSON MEASURES IN ALPHA ORDER 



A CTI C 1 

ArlLc > 


■'EAST74i.ANC 


ANCHQU > 


_ f _ 

■'n 


CATEGS > 


■ 5 


CHARTF « 


» 0 


COOES 1 


■'12345 


DELOU 1 


■'n 


VrlLc 1 


-/ 


DISTRT « 




USTcr > 


■ fi 

■ u 


Cllf\ f T m 


■ on 


FC^^T « 


mf 


T Ft 1 e ■ 
irlLc 1 


"'M741-1.ITM 


I NAMES > 


■ u 


ITEM1 « 


■ 6 


KtTl « 




ICEY2 ■ 


,/ 




|f 


KCVCtiU ■ 

KcTrKn ■ 


■ f\ 

■ U 




I* 1^ 




■ «U1UU 


nri 11* 


^ 2.000 




O AAA 

' 2.000 






nnAUJ ■ 


' 1 .00 




/ 
•» 










iiAuri ■ 
HAnc 1 ■ 


1 




u 






NI 


« 110 


OUTFIT ■ 


0 


PAFILE > 




PANCHQ « 


'n 


POELQU - 


'n 


POFILE > 




PFILE a 


'M741-1.PER 


RCONV m 


3000 


REALSE m 


0 


RESCOR « 




RESFRM m 


0 


Ttnx ■ 


.0 


T811Y ■ 


.0 


TAB3 m 


.0 


TAB4 s 


.0 


TAB567 > 


.0 


TABLES m 


'OOOOIIOIIOOIOC 


TITLE ■ 


'M741-1 


XFILE B 


t 


XWIDE m 


1 



TIME RUM: Jen 30 16:3?:34 1990 
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1 K741-1 "BIGSCAIE" RATING SCALE ANALYSIS VER. 1.53 TABLE 0 

INPUT: 511 PERSONS 110 ITEMS 2 CATEGORIES ANALYZED: 511 PERSONS 110 ITEMS 4An 30 

16:32:34 1990 



CONVERGENCE TABLE 



i HE1H0[> 


ITERATION 


MAX LOGIT CHANGE 
MEASURES STEPS 


MAX SCORE RESIDUAL 1 
MEASURES STEPS | 


PROX 


1 


4.7549 




PROX 


2 


4.405S 




PROX 


3 


.1866 




PROX 


4 


.0471 




UCON 


1 


.0633 


3.85 


UCON 


2 


.0615 


1.55 


UCON 


3 


.0123 


1.49 


UCON 


4 


.0123 


.42 


UCOk 


5 


.0077 


.97 


UCON 


6 


.0045 


.13 



MAX lOGIT CHANGE » MAXIMUM CHANGE IN ANY LOGIT ESTIMATE 
MAX SCORE RESID<>«1 « MAXIMUM DISCREPANCY BETWEEN OBSERVED 
AND EXPECTED SCORES 
MEASURES B PERSONS OR ITEMS 

STEPS « BETWEEN OBSERVED RESPONSE CATEGORIES 
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1 •^BIGSCALE" RATIHG SCALE AHALYSIS 

INPUT: 511 PERSONS 110 ITEMS 2 CATEGORIES ANALTZEO: 511 PERSONS 110 ITEMS 



VER. 1.53 TABLE 5 
Jan 50 16:32:34 1990 



LOGITS 
9.0 

7.2 

5.4 

3.6 



HAP OF PERSONS A>^ ITEMS 
PERSONS--- ITEMS 



1.8 






1.8 


.0 






.0 


-1.8 






-1.8 






XX 








xxxxx 




-3.6 


.« 


XXX 






xx-xxxxxxxxxxxx 


-3.6 






xxxxxxx 








xxxxxxxxx 








xxxxxxxxx 








xxxxxxxxxxx 




-5.4 




xxxxxxxxxxxx 


-5.4 






xxxxxxxxxxioooc 






xxxxxxxxxx 






* .KM 


xxxxxx 








XX 




-7.2 




XXX 


-7.2 






XX 








X 




-9.0 






-9.0 











LOGITS 
9.0 

7.2 

5.4 

3.6 



EACH IN THE PERSON COLUKN IS 4 PERSONS; EACH «.« IS 1 TO 3 PERSONS. 
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VER. 1.53 TABLE 6 
Jan 30 16:32:34 1990 



STATS 



MAP OF ITDH KEAN STEP 



9.0 



7.Z 



5.4 



3.6 



1.8 



.0 



•1.S 



S -3.6 



-5.4 



-7-2 



CONCE 


H^LL 




mm 


BRISK 


EKPHA 


KUDDL 


CLASS 


AGE 


PRECI 


APPRO 


GRIEF 


CABLE 


UITHD 


6UREA 


COMPR 


SHRIV 


GRANT 


GAP 


FLOCIC 


am 


GNAVE 


GRAVE 


EXPLO 


GUIDE 


JOURN 


FIRE 


TOPIC 


SHALL 


COZY 


NURSE 


OUARF 




POOR 


FINIS 


DANCE 


GATE 


KAIL 




KITCH 


GARDE 


STONE 


LITTL 


CHASE 




SIM 







UNSAV DRAW 

SUY FRACT UNIFO BLOI 
POORE CURVA LADY ABOVE 
COHKO DECU VALUE ACT 
GLOBA SECUR AIR PIPE 
SLIDE FEAST ACROS FACE 
YANK ABOUT SLIM NAP 
NATUR JUMP 
PLCS SAVE 



MARKE CAPTA KUD CALL URECK JOB ALARM 



JOKE PLANT BIG 



STORY SLEEP 
EXPU BANK 

FIELD DIZZY JOIN BROOM 
PATH FREEZ EARTH DARK $AODL 
LOVE CHUaC DIVIO MIDDL APART FOLLO OROP PAY 
HANDS SHOOT GLOW TURN BEHIH 



-9.0 



STATS 
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1 M741-1 -BICSCALE" RAT I KG SCALE ANALYSIS 

IHPUT: 511 PERSONS 110 ITEMS 2 CATEGORIES ANALYZED: 511 PERSONS 110 ITEMS 



VER, ',53 TABLE 8 
Jftn T/ 16:32:^4 1990 



I 
T 
E 
N 

I 
H 
F 
1 
T 



10.0 



4.0 



2.0 



.0 



-2.0 - 



-6.0 - 



-8.0 



-10.0 - 



PERSONS 



1 11 

1 i: 1 1 12 
1 1 11 11 

2 11 111 
-2— -1—111 



-7.2 



5.4 



-3.6 



-9.0 


•7.2 


-5.4 -3.6 -1.8 


.0 


1.8 


3.6 


5.4 


7.2 


9.0 






* 1 * 




1 


1 


1 


1 








1 


















1 


















1 11 


















1 

1 












i 
1 


1 

1 1 

1 11 
1 11 1 
1 2 





11 13 1 1 1 1 1 




\1 1 11 1 2 1 1 1 




112 1 




21 1 1 




111 1 1 




11 1 1 1 11 




1 1 1 




1 1 




1 1 1 






1 




1 1 





22134455453211 
11 344791166097442426042 
Q S M S Q 



-1.8 .0 U8 

ITEM CALIBRTN 



3.6 



5.4 



7.2 



■a 
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LJ^l'^ "SIGSCALE" RATIHG SCALE AMALYSIS 

IHPUT: 511 PERSONS 110 ITEHS 2 CATEGORIES AMALY2E0: 511 PERSONS 110 ITEKS 



VER. 1.53 TABLE 9 
Jsn 30 16:32:34 1990 



r 

T 
E 
N 

0 
U 
T 
F 
I 
T 



-9.0 

♦I- 
10.0 - 



8.0 



6.0 - 



4.0 



2.0 - 



-4.0 



-6.0 



-8.0 



-10.0 



PERSONS 



•7.2 

-1- 



-5.4 



-3.6 



•1.8 



1.8 



1 

1 1 1 
1 

1 1 1 



•1- 



1 



1 



3.6 



5.4 



7.2 



1 1 

1 

1 1 1 
1 12 1 
11 1 
1 111 1 12 
2 1 11 

11 111 1 11 
111 1 



•1- 



•1--1- 



11 121 11 
1 121 1 
11 111 

1 1 1 

-2.0 - 11 211 1 1 
1 1 11 1 

11 111 2 1 1 1 
1 1 

1 1 



-5.4 



-3.6 



-1.8 



9.0 



22134455453211 
11 344791 18d09f' ^42426042 

Q S H S Q 



.0 1.8 
ITEM CALI3RTN 



3.6 



5.4 



7.2 
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! aa^^'^.ss "BIGSCALE* RATING SCALE ANALYSIS 

INPUT: 511 PERSCJiS 110 ITEMS 2 CATEGORIES AMALYZEO: 511 PERSONS 110 ITEMS 



VER. 1.53 TABLE 12 
J&n 30 16:32:34 1990 



ITEW* mU STEP STATISTICS ENTRY ORDER 



1.3 
1.1 
1.1 



1^^^ ^ COU«T SAMPLE CALIBRTH ERR«5 KNSQ INFIT | MNSQ OUTFIT WEIGHT |OISPUCE| 

ttlKJCING ITD5S:3 ' 

1 TOPIC 268 509 -5.59A 

2SHAlL0tf 317 510 -5.90A 

3 NURSED 375 509 -6.02A 

4 GAP 174 509 -4.56A 

5 COZY 299 510 -5.68/ 

6 CABLE 108 508 -4.11A 
7eJIDE 276 508 -5.22A 
8 NATim 344 509 -5.65A 
9GHAUE0 222 510 •4.97A 

10 JOURHEY 267 510 -5.35A 

11 COMPRESS 198 510 •4.45A 

12 GRAVEL 327 508 -5.09A 

13 SHRIVEL 283 507 -4.32A 

14 GRANTED 256 510 -4.30A 

15 FLOCK 206 508 -4.56A 
1^ CRAh 242 509 -4.56A 
1/COMHOTIC^ 207 506 -4.16A 
18 DECLARE 227 506 -4.22A 
19UITKDRAU 139 506 -4.02A 
20VALl« 122 506 -4.15A 

21 PRECISE 100 507 -3.75A 

22 APPROPRIATE 108 505 -3.77A 

23 GRIEF 88 504 -3.61A 

24 GLOBAL 237 505 -4.72A 

25 SLAY 143 505 -3.52A 

26 FRACTIOM 115 504 -3.59A 

27 BUREAU 266 504 -4.10A 

28 INFURIATE 129 503 -2.94A 

29 POURED 160 5C4 -3.86A 

30 BRISK 126 504 -2.85A 

31 UNIFORM 178 508 -3.68A 

32 EMPHASIZE 58 507 -2.83A 

33 UNSAVORY 79 505 -2.71A 

34 miDDLE 99 507 -3.33A 

35 CONCEITED 41 506 -2.58A 

36 CURVATURE 216 504 -4.01A 
HEST ITEMS:] 

37 SECURE 237 507 -4.69 

38 DWARF 393 507 -6.19 

39 BLOW 124 506 -3.61 

40 DRAW 68 506 -2.83 

41 POOR 403 504 -6.35 

42 JUMP 362 505 -5.S6 

43 LITTLE 469 508 -7.52 

44 FIRE 301 502 -5.28 

45 AIR 255 505 -4.85 

46 YANK 319 506 -5.42 

47 LOVE 344 506 -5.67 

48 CUSS 101 507 -3.33 

49 CHURCH 341 506 -5.63 

50 MARKET 125 507 -3.62 

51 GATE 430 505 -6.74 

52 ABOUT 311 507 -5.34 

53 GROUHD ZZS 507 -6.10 

54 LADY 160 506 -3.99 .10 1.0 

55 PLUS 398 306 -6.26 .11 .9 



.10 

.10 

.11 

.09 

.10 

.10 

.09 

.10 

.09 

.10 

.09 

.09 

.09 

.09 

.09 

.09 

.10 

.10 

.10 

.10 

.10 

.10 

.11 

.09 

.11 

.11 

.10 

.13 

.10 

.13 

.10 

.13 

.14 

.11 

.14 

.10 

.09 
.11 
.11 
.13 
.11 
.10 
.17 
.10 
.09 
.10 
.10 
.11 
.10 
.11 
.13 
.10 
.11 
.10 

.11 



1.3 
1.1 
1.1 
1.0 
1.1 
.9 
.9 
1.0 
1.0 
1.0 
1.0 
1.0 
1.2 
1.2 
.9 
1.1 
1.1 
1.2 
1.0 
.9 
.9 
.9 
.9 
1.0 
1.3 
1.1 
1.4 
1.7 
1.1 
1.8 
1.2 
1.0 
1.4 
1.0 
.9 
1.1 

1.0 
•9 
1-1 
1.0 
.9 
.9 
.9 
1.0 
1.0 
.9 
1.0 
1.2 
.9 
1.1 
1.0 
1.0 
.9 



6.2 
2.7 
.9 
.4 
1.2 
-3.6 
-2.0 
.4 
-.4 
-1.0 
-.7 
.7 
4.9 
6.0 
-4.9 
2.0 
2.9 
5.8 
-.7 
-3.4 
-1.8 
-2.4 
-2.3 
-.1 
4.2 
1.4 
9.8 
6.5 
2.1 
7.2 
3.8 
-.5 
3.3 
.6 
-1.3 
2.6 



-1.8 

-1.5 
1.5 
.5 

-1.0 

'2.5 
-.9 
1.3 
1.7 

-2.7 
-.5 
3.2 

-3.8 
2.3 
-.1 
-.9 

-2.2 
1.2 

-2.0 



1.0 
1.0 

.9 

.9 
1.0 
1.0 

.9 
1.0 
1.0 
1.2 
1.3 

.9 
1.1 
1.2 
1.3 
1.0 

.9 
1.0 

.9 
1.0 
1.0 
1.5 
1.3 
1.5 
2.1 
1.2 
2.5 
1.2 
1.2 
1.8 
1.3 
1.1 
1.1 

1.0 
.8 
1.1 
1.2 
.8 
.3 
.7 

1 

1.1 

.9 
1.0 
1.7 

.8 
1.3 
1.1 
1.0 

.8 
1.1 

.8 



5.4 
1.5 
.8 
.2 
.7 
-2.7 
-2.1 
-.3 
-.2 
-1.3 
.6 
.9 
5.1 
6.6 
-3.8 
1.9 
3.2 
6.0 
-.1 
-1.9 
-.3 
-1.4 
-.4 
-.3 
5.1 
3.8 
9.0 
6.9 
2.7 
8.7 
2.6 
1.5 
5.0 
2.6 
.5 
2.5 

•1.1 
-2.5 
1.6 
1.8 
-1.9 
-2.9 
-1.9 
.9 
1.5 
-2.8 
-.8 
6.6 
-4.2 
3.9 
1.2 
-.7 
-2.7 
1.1 
-2.3 



.00 
.00 
.00 
.01 
.00 
.02 
.01 
.00 
.01 
.00 
.01 
.01 
.01 
.01 
.01 
.01 
.02 
.01 
.02 
.02 
.02 
.02 
.03 
.01 
.03 
.03 
.02 
.05 
.02 
.06 
.03 
.06 
.07 
.04 
.08 
.02 

.01 
.00 
.03 
.06 
.00 
.00 
.00 
.01 
.01 
.00 
.00 
.04 
.00 
.03 
.00 
.00 
.00 
.02 
.00 
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I m 



NAME 



56 SLIH 

57 FINISH 

58 CAPTAIH 

59 DIVIDE 

60 PXPE 

61 !MVE 

62 NAP 

63 MIDDLE 

64 PATH 

65 FREEZE 

66 APARTMENT 

67 RUN 

68 MUD 

69 DANCE 

70 MAIL 

71 ABOVE 

72 EXPUIN 

73 EXPLORE 

74 SLIDE 

75 BANK 

76 JOKE 
7? FOLLOW 

78 HANDSOME 

79 CHASE 

80 EARTH 

81 CALL 

82 FEAST 

83 DROP 

84 DARK 

85 PAY 

86 ACROSS 

87 KITCHEN 

88 PLANT 

89 SHOOT 

90 GARDEN 

91 GLOW 

92 ACT 

93 FACE 

94 AGE 

95 WRECK 

96 SADDLE 

97 TURN 

98 JOB 

99 STORY 

100 FL;LO 

101 DI^'.ZY 

102 ALARM 

103 B'.G 

104 fTOHE 

105 JOIN 

106 HALL 

107 SLEEPY 

108 SROOH 

109 DRY 

110 BEHIKD 



327 
<'>2 
137 

230 
382 
301 
350 
327 
335 
361 
484 
115 
422 
431 
167 
216 
259 
266 
225 
409 
344 
399 
463 
299 
128 
267 
359 
313 
354 
272 
445 
412 
^92 
460 
390 
184 
259 

96 
121 
318 
391 
132 
187 
257 
271 
112 
412 
460 
265 

47 
175 
256 
351 
372 



SAMPLE 


CALieRTN 


ERROR 


MNSQ 


INFIT 


1 MNM OUTFIT 


WCIUn 1 


505 


-5.50 


.10 


.0 


•4.3 


0 

.0 


•I.J 


flfl 


S^5 


-6.61 


.12 


.9 


•1.9 


7 




flfl 


503 


•3.76 


.10 


1.2 


3.6 


1 5 
1 .J 




fl9 


307 


-5.62 


.10 


1.0 


-,5 


1 0 


.0 


flfl 


506 


•4.63 


.09 


.9 


•3.4 


0 

.T 


c.y 


fli 

.U 1 


507 


•6.06 


.11 


,9 


•1.4 


0 


• 1 0 

1 .y 


flfl 


506 


•5.26 


.09 


1.1 


3.9 




L n 


fli 

V 1 


506 


•5.72 


.10 


,9 


•2.9 


ft 
.0 


mX fl 


flfl 

.vv 


507 


•5.49 


.10 


.8 


•4.9 


ft 


*.y 


OA 


507 


•5.57 


.10 


1.0 


1.1 


1 . 1 


1.9 


flfl 


505 


•5.85 


.10 


1.0 


-.7 


1 n 




flfl 


506 


•8.14 


.22 


,9 


•.4 


.0 


-1 0 
1 .y 


flfl 


505 


•3.51 


.11 


1.0 


-.5 


I.I 


7 




507 


•6.59 


.12 


,9 


•1.3 


ft 

.0 




flfl 


506 


•6.74 


.13 


1.0 


-.2 


1 1 

1 . 1 


1 . 1 


flfl 


505 


•4.07 


.10 


1.1 


2.5 


1 . 1 


c.o 


fl9 


505 


•4.52 


.09 


,9 


•2.3 


1 n 


• y 
•. f 


51 

.Ui 


506 


•4.89 


.09 


1.0 


•.9 


1 n 


•.3 


fli 


502 


•4.96 


.09 


1.0 


-,2 


1 


.3 


fli 

.U 1 


505 


•4.60 


.09 


1.3 


8.6 


1 % 


ft 9 


fli 

.U 1 


506 


•6.41 


.12 


1.0 


.3 


1 n 


• 4 
.1 


flfl 


506 


•3.67 


.10 


1.0 


•.6 


0 
.y 


1.1 


00 
. vv 


506 


•6.28 


.11 


1.1 


1.2 


1 1 
I.I 


1 .J 


flfl 
. vv 


505 


•7.43 


.16 


.9 


•.5 


.0 


•1 •H 


flfl 


502 


•5.26 


.09 


.9 


•3.6 


0 
.y 


•t 7 


ni 


505 


•3.66 


.11 


1.0 


.3 




C.I 


fl'X 


503 


•4.97 


.09 


.9 


•2.5 


Q 

.y 


-9 T 


ni 


506 


•5.82 


.10 


1.0 


.0 


1 .VF 


/ 
.H 


flfl 
.uu 


504 


•5.38 


.10 


,9 


•2.3 


0 


-9 ft 


flfl 
.vv 


506 


•5.77 


.10 


1.0 


•.3 


1 .u 


, 7 


flfl 

. vU 


504 


•5.01 


.09 


1.1 


?.4 


1 1 
1.1 


9 % 

C.J 


fli 

.VI 


502 


•7.06 


.14 


,9 


•.7 


ft 
.0 


-1 7 


flfl 

.Vv 


504 


•6.47 


.12 


1.0 


.3 


1.1 


1 C 
1.3 


flfl 
. VV 


503 


•6.21 


.11 


.9 


•1.2 


,9 


-1 0 

1.7 


flfl 
.vv 


504 


•7.37 


.16 


.9 


-.7 


.6 


-9 L 


.00 


498 


•6.23 


.11 


.9 


•1.2 


,9 


-1 5 
I.J 


.00 


501 


•4.24 


.10 


1.1 


2.2 


1.1 


1.0 


.01 


505 


•4.89 


.09 


1.0 


.9 


1.0 


1 0 
1 . u 


.01 


502 


•3.2n 


.12 


1.0 


•.5 


1.1 


.8 


.04 


505 


•3.58 


.11 


1.1 


1.4 


1.2 


2.6 


.03 


504 


•5.43 


.10 


1.0 


.8 


1.0 


.5 


.00 


504 


•6.20 


.11 


1.0 


-.7 


.9 


. 7 


.00 


503 


-3.71 


.10 


1.1 


2.0 


1.3 


4.1 


.02 


505 


•4.26 


.10 


1.0 


-.2 


1.0 


.3 


.01 


506 


•4.87 


.09 


1.0 


.0 


1.0 


•.3 


.01 


CAE 

503 


•5.00 


.09 


.9 


-3.0 


.9 


-3.0 


.01 


506 


•3.47 


.11 


1.1 


1.7 


1.3 


2.9 


.03 


505 


•6.46 


.12 


.9 


•2.2 


.7 


-2.9 


.00 


505 


•7.35 


,16 


.9 


•1.2 


.5 


-3.2 


.00 


503 


•4.95 


.09 


.9 


•S.2 


.9 


-2.9 


.01 


504 


•2.41 


.15 


1.1 


.8 


1.7 


3.8 


.09 


504 


•4.14 


.10 


.9 


•1.6 


.9 


-1.1 


.02 


504 


•4.87 


.09 


1.0 


1.3 


1.0 


1.3 


.01 


503 


•5.75 


.10 


1.0 


.6 


1.1 


1.6 


.00 


505 


•5.96 


.11 


.9 


•1.6 


.9 


-1.9 


.00 



"WEIGHT-'S ARE KULTIPLICATIVE ON A RATIO SCALE. "CALIBRTN"S ARE ADDITIVE OH AH IHTEKVAL SCALE. 

THE STANDARD ERROR OF A WEIRHT IS THE VALUE OF THE "UEIGHT" TIMES THE VALUE OF THE CALIBRTH "ERROR" 
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-BIGSCALE'' RATIMG SCALE ANALYSIS 
IHPUT: 511 PERSONS 110 IT5HS 2 CATEGORIES ANALYZED: 511 PERSONS 110 ITEMS 



VER. 1.53 TABLE 20 
Jan 30 16:32:34 1990 



SUMMARY or 511 IffiASURED """RSONS 



1 


COUNT 


TEST 


KEASURE 


ERROR 


NNSQ 


INFIT 


KNSQ OUTFIT | 


1 l£AN 


56.7 


108.8 


-4.88 


.23 


1.0 


.1 


1.1 .2 1 


1 


14.3 


7.5 


.70 


.02 


.2 


1.4 


.3 1.4 1 



I RHSE .23 AOJ.S.D. .66 PERSON SEP 2.89 PERSON SEP REL. .89 
SUmARY OF 110 CALIBRATED ITEMS CENTERED ON MEAN STEP VALUE 



1 


COUNT SAMPLE 


CALIBRTN ERROR 


MNSQ 


INFIT 


MNSQ OUTFIT | 


1 MEAN 
1 S.D. 


263.6 505.6 

115.7 2.1 


-4.96 .11 
1.22 .02 


1.0 
.1 


.2 
2.7 


1.1 .5 1 
.3 2.9 1 




1 RMSE 


.11 AOJ.S.D. 1.22 


ITEM SEP 


11,14 


ITEM SEP 


REL. .99 1 



SUMMARY OF CALIBRATED STEPS 



1 LABEL 


VALUE 


COUNT 


MEASURE 


ERROR RESIDUAL | 


1 ? 


0 


26622 


NONE 


3.4 1 




1 


28995 


NONE 


-3.4 1 



OUTFIT: MEAN SGUARE STANDARD RESIDUAL -- STANDARDIZED TO (0,1) EXPECTATICW 
INFIT: MEAN SOUARE INFORMATION RESIDUAL - STANDARDIZED TO (0,1) EXPECTATIOTI 

SEPARATION: RATIO OF ADJUSTED SD TO ROOT MEAN SQiiARE ERf^CR 
RELIABILITY: RATIO OF ADJUSTED VARIANCE TO OBSERVED VARIANCE 
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Appendix G 
Database Structure 



Definitions of StnictuK; Tenns: 



Field 

Field Name 
Type 



Width 



Dec 



Nth variable in the database record 
Name of the field 

The type of field. "Character" includes alphanumeric data; 
"Numeric" includes only numbers; "Logical" values are "T" for 
Trup- or "F" for False; "Memo" contains unlimited alphanumeric 
data ill a word-processing format. 

The numbers of places held for data within that field. Logical 
fields always contain 1 space. Memo fields are listed as containing 
10 spaces but are actually variable, dependmg on the numtie: of 
characters entered. 

For numeric fields, this is the number of places to the right of the 
decimal place. For other fields, vi/s is irrelevant. 
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Descriptions of Major Fields 



CATl 
CAT2 
CAT3 
CAT4 
CAT5 

CHOICBl 
CH0ICE2 
CHOICES 
CH0ICE4 
CHOICES 

CURRENT 



DIFF 



DISCUSSION 

ERROR 

EXl 

EXIANS 
EX2 

EX2Ar« 



The type of choice of the first through fifth item choice. 
For the most part, the synonym is marked as "s," and the 
other choices not markal. Beginning with the Worksample 
735 test series, all choice types are identified (synonym, 
sound-alike, close mislead, same situation, antonym). 

The text of the five item choices. 



Whether the given item administration is the admmistration 
that was ultimately used to estimate VSS or was superseded 
by a later administration (for each meanmg of a word). 
Current is only true once for each meaning of a word, even 
though the word may have been used in several different 
items, each of which may have been administered on 
multiple occasions. 

A difficulty rating of the word on the scale of 1 to 5 
originally devised by Gary Supanich to subjectively estimate 
the difficulty of the word. The use of Tne Living Word 
Vocabulary (Dale & O'Rourke, 1981) makes this value 
obsolete. 

Wordbook discussion. 

The standard error of the item's logit measure. 
Wordbook Bxtxcist 1. 
Wordbook Fjcercise 1 answer. 
Wordbook Exercise 2. 
Wordbook Exercise 2 answer. 
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EX3A 

EX3B 

EX3C 

EX3ANS 

FORMl 
F0RM2 
F0RM3 
F0RM4 
FORMS 

GLOBMEAS 
GOODSAMPLE 

GOODQUAL 

INFIT 
ITEM 

ITEMWORD 
LINK^ANCHR 

MEANING 



Wordbook Exercise 3 Choice A. 
Wordbook Exercise 3 Choice B. 
Wordbook Exercise 3 Choice C. 
Wordbook Exercise 3 answer. 

First through fifth Wordbook alternative form of the word. 



The VSS value of the word expressed in logits (50% 
correct). 

Whether the overall ability of the sample was appropriate for 
the difficulty level of the item. (See Research Memorandum 
1990-2 for complete details.) 

Whether the item is of good quality relative to the values of 
INFIT, OUTFIT, and their related mean squares. (See 
Research Memorandum 1990-1 for details.) 

The INFIT value. 

The item number of the item within the test. 
The tested word. 

Whether the item is a linking item, an equating item, or 
neither. 

A now-obsolete code that refers to the numerical code of the 
meaning tested by the Foundation (the new system will list 
the word and its synonym). 



MEANSQ 



The mean square INFIT value. 
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MEASURE 



OUTFIT 
PARTSPEECH 

PHRASE 

REVIEW 

REVIEWANS 

REVISION 

ROOTWORD 

SAMPLE 

SCORE 

SPEECHl 
SPEECH2 
SPEECH3 
SPEECH4 
SPEECH5 

TEST 

VSS80 

WEIGHT 



The logit nieasure of the item relative to the &' uple, not 
necessarily anchored to Worksample 705-1. l,ew items are 
automatically anchored, and so the MEASURE field is the 
same as GLOBMEAS. 

The OUTFIT value. 

A one-character code giving the word's part of speech 
(n=noun; v=verb; a=adjective). 

Thi* phrase in which the word was tested. Our testing 
program no longer uses phrases. 

Wordbook review item. 

Wordbook review item answer. 

The number of the revision of a given word for the given 
meaning (e.g., 1st revision, 2nd revision, and so on). 

The ITEMWORD stripped of suffixes and prefixe.';. 

The number of people who took the item. 

The number of persons who answered the item correctly. 

Part of speech of the first through fifth alternative forms of 
the word. 



The worksample number for the test item (e.g., "705-1"). 

The VSS value of die word. 

The statistical "weight" of die item. 
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Structure of ITEMS.DBF 
Number of data records: 6943 



Date of last update: 06/08/89 



JlIwIQ 




lype 


Widtn 


1 
1 


lijcJM.VVUxvU 


unaracter 


20 


9 


lYlIl/vLNJLXNVJ 


L/iiaracicr 


t 
1 






numeric 


t 
1 






iMumGnc 


t 
1 






I'M n aaM JM.— 




c 

0 




f 'ruiTi i->4 fit 

L/iiEracicr 


1 


7 






45 


Q 
O 




L/naiHcicr 


OA 


9 


CATl 


Character 


1 


10 


CH0ICE2 


Character 


20 


11 


CAT2 


Character 


1 


12 


CHOICES 


Character 


20 


13 


CAT3 


Character 


1 


14 


CH0ICE4 


Character 


20 


15 


CAT4 


Character 


1 


16 


CHOICES 


Character 


20 


17 


CAT5 


Character 


1 


♦♦Total 






195 
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structure of USED.DBF 



Number of data records: 4002 
Date of last update: ll/24"»7 

Field Field Name Type Width Dec 

1 TEST Character 6 

2 ITEM Numeric 3 

3 ITEMNAME Character 20 
** Total ** 30 
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structure of DISCUSS.DBF 



Number of data records: 1000 
Date of last update: 11/20/89 



Fielo 


Field Name 


Type 


1 


kOOTWORD 


Character 


2 


MEANING 


Character 


3 


EXl 


Character 


4 


1 A XTC 

EXIANS 


Logical 


5 


EX2 


Character 


0 


EX2ANS 


Logical 


7 


EX3A 


Character 


o 
o 


EX3B 


Character 


9 


EX3C 


Character 


lu 


EX2ANS 


Chamcter 


11 


REVIEW 


Character 


Iz 


REVIEWANS 


Character 


13 


TNT C* T C* C*T/^XT 

DISCUSSION 


Memo 


14 


FORMl 


Character 


15 


SPEECHl 


Character 


16 


F0RM2 


Character 


17 


SPEECH2 


Character 


18 


F0RM3 


Character 


19 


SPEECH3 


Character 


20 


F0m4 


Character- 


21 


SPE'£CH4 


Character 


22 


F0RM5 


Character 


23 


SPEECH5 


Character 


**Total 







Width Dec 
25 

1 
75 

1 

150 

1 
25 
25 
25 

1 

200 
25 
10 
25 

8 
25 

8 
25 

8 
25 

8 
25 

8 

730 
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Structure of ALLSTATS.DBF 



Number of data records: 6257 
Date of last update: 03/06/90 



Field 


Field Name 


Type 


Width 


Dec 


1 


WORD 


Character 


16 




2 


TEST 


Character 


6 




3 


HEM 


Numeric 


3 




4 


VSS80 


Numeric 


4 




5 


GLOBMEAS 


Numeric 


5 


2 


6 


SCORE 


Numeric 


3 




7 


SAMPLE 


Numeric 


3 




8 


WEIGHT 


Numeric 


4 


2 


9 


MEASURE 


Numeric 


5 


2 


10 


ERROR 


Numeric 


4 


2 


11 


MEANSQ 


Numeric 


3 


1 


U 


OUTPrr 


Numeric 


4 


2 


13 


INFTT 


Numeric 


4 


2 




LINK ANCHR 


Logical 








GOODSAMPLE Logical 






16 


GOODQUAL 


Logical 






17 


CURRENT 


Logical 






18 


TEMP 


Logical 







** Total ** 70 
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Structure of STATS.DBF 



Number of data records: 8761 



Date of last update : 06/22/88 



Field 


Field Name 


Type 


Width 


1 


imiWORD 


Character 


25 


2 


REVISION 


Numeric 


1 


3 


TEST 


Character 


6 


4 


rraM 


Numeric 


3 


5 


LINKING 


Logical 


1 


6 


WORDBOOK 


Numeric 


2 


7 


VSS80 


Numeric 


3 


♦♦Total 


** 
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Appendix H 
Test Series Contained in Each Database 



Test Senes Contain^J in USED.DBF 





722S 




722F 




722G 


705-1 


722H 


705-10 


7221 


70J-U 


722J. 


705-2 


735A 


705-3 


735B 


705-4 


738 series 


705-5 


Wordbook 1 


705-6 


Wordbook 2 


705-7 


Wordbook 3 


705-: 


WordUok 4 


705-9 


Wordbook 5 


722A 


Wordbook 6 


722B 


Wordbook 7 


722C 


Wordbook 8 


722D 
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Test Series Contained in ALLSTATS.DBF 



oyurv 


734A* 




735A 


fiOflT 


735B 




735C 


7fK.1 


735D 


/UJ-'XU 


735E 




738A 




738B 




738C 




73oD 




7jo£ 


705-6 


/jor 


705-7 


liovj 


705-8 


/JOil 


/Vol ✓ 


IDOL 


708A 


/jcJ 


7CoB 




708C 




722A 


/JoM 




Wordbook 1 


722C 


Wordbook 2 


722D 


Wordbook 3 


722E 


Wordbook 4 


722F 


Wordbook 5 


722G 


Wordbook 6 


722H 


Wordbook 1 


7221 


Wordbook 8 


722J 
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Test Series Contained in STATS.DBF 



176AB 


687 


176AD 


687B 


176BA 


698A 


180AD 


699-1 


180AE 


699-10 


180AF 


699*11 

XX 


180BA 


699-12 


I80BB 


6^-13 

yj^j^ X*/ 


271B 


6^-14 


271C 


6^-15 

yj^^ X*/ 


600-A 


699-16 

yj^^ XV/ 


600AA 


699-2 


6C0AB 


699-3 


600C 


699-4 


603F 


699-^ 


604-E 


699-6 


605-BA 


699-7 


605-CA 


699-8 


605-FA 


699-9 


620D 


702 


620E 


704 


620F 


708A 


629A 


708B 


629AA 


708C 


629AC 


734A* 


629B 


95AD 


641CA 


95BC 


641CC 


95CC 


641DC 


95DB 


641EA 




641EC 


95GA 


641F 


95H 


641G 


951 


649A 


95JB 


678A 


GINNB 


680A 


GINNC 


680B 


GINNF 


684A 
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Appendix 16 
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